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Introduction 


“Cognition in Space Workshop I: Metrics and Models” was the first in a series of workshops 
sponsored by NASA to develop an integrated research and development plan supporting human 
cognition in space exploration. The workshop was held in Chandler, Arizona, October 25-27, 
2004. The participants, who represented academia, government agencies, and medical centers, 
are identified in Appendix A. 

The workshop addressed the following goal of the NASA Human System Integration (HSI) Pro- 
gram for Exploration: Develop a program to manage risks due to human performance and human 
error, specifically ones tied to cognition. Risks range from catastrophic error to degradation of 
efficiency and failure to accomplish mission goals. “Cognition” includes memory, decision 
making, initiation of motor responses, sensation, and perception. 

Four sub goals were defined as follows: 

1 . NASA needs to develop a human-centered design process that incorporates standards for 
human cognition, human performance, and assessment of human interfaces. 

2. NASA needs to identify and assess factors that increase risks associated with cognition. 

3. NASA needs to predict risks associated with cognition. 

4. NASA needs to mitigate risk, both prior to actual missions and in real time. 

This report develops the material relating to these four subgoals. 


1. NASA needs to develop a human-centered design process that incorporates standards 
for human cognition, human performance, and assessment of human interfaces. 

To successfully support cognitive performance, the crew’s roles and responsibilities as well as 
their training, schedules, environment, and system interfaces must be integrated into mission and 
vehicle designs from the beginning. During the mission conceptual design, conscious decisions 
on function allocation between the crew and the software and hardware must be based on princi- 
ples of human performance. Current knowledge about human cognitive performance provides 
critical requirements for scheduling and workload. In turn, these requirements impact crew 
selection criteria and crew size. 

The system design includes not just the onboard crew and systems but also the ground support 
and the communications systems and protocols that link them. A “lesson learned” from the In- 
ternational Space Station is that simply standardizing words and phrases - limiting the critical 
vocabulary - contributes significantly to efficiency. Cognitive performance of ground crews 
must be sustained. 

Procedures and training materials are two tools that can support crew cognitive performance. The 
human centered design process should develop these to be highly effective. To this end, the HSI 
program should develop requirements and guidelines for procedure and training material writing 
and presentation based on current knowledge and best practices. The HSI Program does not 
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currently have expertise in this area; expanding the program to address this is one important 
need. 

Recommendation 1: Expand the HSI Program to systematically develop requirements and 
guidelines to enhance procedure and training development. 

The human centered design process must incorporate medical requirements. With respect to 
cognition, medical requirements address assessment and diagnostic tools for cognitive states, 
tools to identify causes of cognitive changes, and countermeasures. Schedules and workload, 
fatigue, medications, environmental factors, atmospheric contaminants, and social factors all 
potentially impact cognitive performance. The effects of poor or inadequate sleep and circadian 
rhythm are primarily manifested in cognitive performance. 


2. NASA needs to identify and assess factors that increase risks associated with cognition. 

The first steps in managing risks are identifying the contributing factors and root causes, and 
assessing the magnitude of their impacts. What performance-shaping factors are most important 
to cognition? Thinking of the flight crew, the ground crew, and the spacecraft as a system, it is 
necessary to identify the feedback and control loops. What data are required to provide feed- 
back on the state of the system? What factors can be changed to reestablish a satisfactory 
system state? 

Generally, introspection or self report is unreliable as a data source. On the other hand, flight 
and ground crew members, especially the crew medical officer and the flight surgeon, have an 
important role as system observers. Various other methods to assess cognitive performance must 
be evaluated and policies determined. Some data collection methods to consider include: 

1 . embedded measures, 

2. biomarkers, 

3. formal assessment tools (e.g., WinSCAT, Mini-Cog), 

4. opportunistic data collection (observational/video, voice analysis), 

5. synthetic task environments, 

6. performance evaluation in competitive context, and 

7. mission simulations in trainers. 

To assess the sensitivity of the system to decrements in cognition requires understanding the 
tasks to be performed and the roles of the humans. Systematic analysis of prior experience can 
contribute to developing error trees or risk trees. Root cause analysis of lessons learned from 
prior incidents leads to principles that can be applied in future system design. Controlled 
studies are required to assess relative importance of factors. 

Quantitative risk assessment for human performance is an emerging discipline. The field of 
behavioral economics can contribute to understanding risk assessment and judgment processes. 
This is an area that has not been reviewed by NASA human factors or behavior and performance 
programs to date. 


2 



Recommendation 2: Commission a report of the literature on human risk analysis and its 
applicability to space flight. 

A wide range of factors can influence the risks of poor performance. Some of them include 
environmental factors, psychosocial factors, fatigue, individual differences including reaction to 
risk factors, stress, and workload intensity and duration. Controlled studies to assess effects of 
individual and combinations of factors are needed. 

Recommendation 3: Systematically assess factors contributing to cognitive decrements and 
their interactions. 

A variety of assessment tools exist. The first step in selecting or tailoring assessment tools is to 
relate the measurements to specific performance objectives (tasks). Determine which capabilities 
require new assessment tools or additional tests within existing batteries. Prospective memory and 
attentional blindness were identified as two aspects of cognition not covered by existing tests. 
Determine the necessary sensitivity for assessment tools. Consider that international teams will 
be using these tools, which poses linguistic and cultural challenges to selecting assessment tools. 

There are two main effects to detect: trauma and disease. When decrements are detected, the 
cause must be identified to enable corrective or ameliorative action. (Corrective action restores 
the system to the desired state; e.g., reducing the carbon dioxide concentration. Ameliorative ac- 
tion enables the system to continue with degraded capabilities by compensating for them; e.g., 
omitting the most difficult tasks; relying more heavily on automation.) Diagnostic tools are im- 
portant for assessing effects of events such as fires, spills of toxins, etc. Specific performance 
predictor tests are important to assess readiness to perform for critical tasks. 

There is a literature on measures of attention that are predictors of performance in real-world 
tasks, particularly driving. The literature on vigilance and fatigue addresses individual perform- 
ance over time rather than just group means. Measures of attention and situational awareness 
need to be assessed for relevance to individual performance in space operations. 

Some measures of performance require a signal detection theoretic approach: what are the 
tradeoffs between speed and accuracy? How sensitive are critical tasks to speed and accuracy, 
and how consciously do crew members make these trades? The extravehicular activity slogan, 
“the slower you go, the sooner you finish,” reflects crew awareness of speed and accuracy 
tradeoffs in motor skills. Does this extend to cognitive performance? 

Recommendation 4: Review the literature on predictors of individual performance in real- 
world tasks and develop a set of tests to be used during all NASA-funded research simulating 
space missions (e.g., bed-rest studies). 

Recommendation 5: Develop a suite of tasks researchers can use to measure the validity of 
their results. 

A decision tree assessment strategy must be developed to provide short tests with rapid feedback 
to determine whether there are significant performance decrements, and more detailed tests to be 
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used sequentially to identify causes. This minimizes crew time wasted on testing, while provid- 
ing useful information when problems are identified. A testing protocol needs to be developed in 
this context: how often are routine tests needed? What situations, such as docking maneuvers, 
require special readiness-to-perform assessments? What tests are indicated for identifying 
specific problems? 

An assessment standard must also address performance norms. For any specific task, what 
aspects of cognition are critical? What are acceptable operating bands? To what extent are the 
operating bands based on individual baseline date rather than group norms? 

Recommendation 6: Develop processes for identifying the critical aspects of cognitive 
performance that are essential or limiting for specific critical tasks. ( What are the most 
important cognitive performance features to support?) 

Figure 1 provides a conceptual illustration of the steps that need to be accomplished to 
incorporate assessment of cognitive performance into operations. 


3. NASA needs to predict risks associated with cognition. 

Cognitive models can be used to predict risks associated with various tasks. Models of cog- 
nition are needed for the following purposes: (1) analyze sensitivities to cognitive decrements 
for task design and mission planning; (2) analyze sensitivities to environmental factors, training, 
schedules, and workload; (3) estimate performance parameters such as time to perform and error 
rate for planning and for trade studies; (4) compare alternate designs for time and error rates; 

(5) determine whether designs meet functional requirements for time and errors. 

Three types of models based on the nature of the models are (1) regression analysis and 
factor analysis, based on primarily empirical analysis; (2) outcome and predictive modeling; 

(3) cognitive process modeling, based on analyzing tasks into components (e.g., access working 
memory, recall from long-term memory, visual search) and assembling submodels with associ- 
ated times and errors. Cognitive models can focus on modeling the effects of performance mod- 
ulators (e.g., fatigue, g-forces, pharmaceuticals) which can be applied to models of a variety of 
tasks. Models can predict “normal” or “average” performance; stochastic models can predict 
ranges of performance; some models focus on predicting individual differences on various 
tasks based on responses to, for example, fatigue on a specific task. 

Models of performance in space flight are limited by access to data. There are significant issues 
in collecting data on cognitive performance in space. Existing models are based on data acquired 
from higher or lower fidelity analogs, including military high-performance aircraft, commercial 
aviation, and driving. Data acquisition during flight and training is strongly resisted. 

Much of the best work in cognitive modeling of high-performance systems is carried out by the 
military. This should be leveraged to benefit exploration planning. 
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Recommendation 7: Hold a joint meeting of NASA cognitive scientists with the DARPA 
[Defense Advanced Research Projects Agency] Augmented Cognition Program Manager. 

Recommendation 8: Develop a strategic plan relating modeling to operational planning. 

Figure 2 illustrates the process by which human performance models may contribute to trade 
studies resulting in operational decisions. They provide a means to estimate effects of design 
and operational decisions on mission outcome. 


4. NASA needs to mitigate risks, both prior to actual missions and in real time. 

Risks due to human cognitive performance degradation or failure can be managed by a 
systematic approach to designing cognitive tasks and to maintaining human performance. 

Pre Mission 

The process begins by designing the tasks needed to accomplish the mission objectives to 
incorporate human capabilities and limitations. Determine the functions to be accomplished and 
allocate the functions among the crew, the ground support, and the automated systems. Design 
the interfaces between the users and the hardware and software systems to enable the humans to 
perform the functions they do best - pattern recognition, creative problem solving, integration of 
novel information - and to assign as little of the rote tasks to the human as possible. Procedures 
and tools similarly need to be designed to facilitate crew performance. 

Human performance is affected by an individual’s physical environment in a variety of ways. 
Architectural factors such as the layout of the spacecraft, lighting, sound proofing, and volume 
allocation affect performance directly or indirectly. Such factors as food preparation options and 
menu variety can indirectly affect cognition through the effects on crew health of good nutrition. 
Toxins and contaminants in the environment potentially damage the crew physically, including 
neurologically, which may result in cognitive decrements. 

Crew selection and training can also mitigate risks of cognitive decrements. One example is 
that individuals differ in their ability to maintain performance during sleep deprivation. Identi- 
fying such individual differences for use in crew selection may be a way to reduce the effects 
of irregular sleep schedules. Training can enhance knowledge and application of fields such as 
medicine, robotics, and navigation. Mental skills such as decision making, risk estimation, or 
sustained attention can be strengthened through training. NASA should develop protocols for 
just-in-time training and retraining based on expected and observed learning decrements 
across time. 

Real Time 

During the mission, countermeasures can be applied to maintain cognitive performance at pre- 
mission levels, or to return it to baseline levels after a decrement. Some ways to accomplish this 
include training, including refresher training; procedure design to be adaptive to crew needs and 
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performance levels; and dynamic task allocation. These may require embedded measures to de- 
tect the necessity for a change in the level of automation or level of prompting in procedures. 

Pharmacological countermeasures may be used: stimulants can enhance performance for fatigued 
individuals; sedatives can enable sleep to restore performance. Other pharmacological measures 
need to be investigated for efficacy, side effects, and interactions. 

Scheduling must be adjusted to accommodate cognitive performance. As mentioned above, 
studies of tradeoffs between time and accuracy indicate that some tasks may be performed ade- 
quately well if they can be done more slowly. Schedule adjustments to accommodate refresher 
training, or rest for overly fatigued personnel, can maintain or restore performance. 


Conclusion 

We extend our thanks and appreciation to the participants in the “Cognition in Space 
Workshop I: Metrics and Models” who gave their time and expertise to help HSI/NASA begin 
planning for a rigorous and accountable research program on the risk management of cognitive 
human performance and error. All participants were subject matter experts in and outside of the 
government in how to measure and model cognition and performance. A list of the participants 
is found in Appendix A. The workshop generated eight recommendations from four sub goals: 
development of a human-centered design process that incorporates standards for human cog- 
nition, human performance, and assessment of human interfaces; identification and assessment 
of factors that increase risks associated with cognition; prediction of risks associated with 
cognition problems; and cognitive decrement risk mitigation. 

The workshop leaders will communicate and encourage required funding and implementation 
of the eight recommendations to Human Systems Integration, Human Research & Technology 
Directorate at NASA Headquarters. If approved, some of the recommendations can be 
implemented immediately, such as reviews of the relevant literature (recommendations 2 and 4) 
and setting up a meeting with the DARPA Augmented Cognition Program Manager 
(recommendation 7). Assessment of cognitive performance and related factors as well as 
consistent validation procedures will be multiyear processes that will need to be annually 
reviewed for progress and appropriate alignment with operational goals (recommendations 3,5, 
and 6). Developing a strategic plan relating modeling to operational planning and expanding the 
HSI Program to systematically develop requirements and guidelines to enhance procedure and 
training development will be ongoing reiterative processes, based on completion of other 
recommendations and dialogue with all involved stakeholders (recommendations 1 and 8). 

In addition to following through on the recommendations listed above, future activities include a 
second workshop on risk mitigation and requirements and continued dialogue with stakeholders, 
NASA centers, and the broader research community. 
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Evidence path for cognitive testing in space flight 



Figure 1 : Evidence path for cognitive testing in space flight, (credit: David Dinges, National 
Space Biomedical Research Institute) 
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Path for incorporating modeling in operational planning 



Figure 2: Path for incorporating modeling in operational planning. (Product of workshop 
participants.) 
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